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(54) Abstract Title 

Vehicle head-up display control 

(57) In a vehicle having a head-up display, virtual images 6 of control elements, such as switches, are 
displayed and the functions associated with the virtual images are operated by the detection of the position of 
the index finger 4 of the driver within an operating area 3. The finger position may be detected by an image 
analyser receiving an image from a camera 2 or the operating area 3 may include a touch pad A loudspeaker 
may signal the successful actuation of a control element 6 and output an acoustic warning if the hand of the 
driver leaves the operating area 3. The image of the control element also changes to indicate that it has been 
actuation. ^ 
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CONTROL DEVICE 

BACKGROUND OF THE INVENTION 

The invention relates to a control device, particularly for minimum-distraction 
operation of switches in a vehicle. 

From German Patent DE 196 53 595 CI a control device is known which 
enables the driver of a vehicle to operate switches without the driver having to 
take his eyes off the road. For this purpose, this control device has a camera 
which is directed to the switches to be operated and thus takes a picture of the 
switches and of the driver's hand. This picture is projected into the driver's 
field of vision by means of a "head-up display", so that the driver can 
determine the position of his hand relative to the switches to be operated, 
without taking his eyes off the road. Equipment within the vehicle (e.g. climate 
control, radio, navigation system, telephone) can be operated in this way 
without endangering road safety. Furthermore, from the aforementioned patent, 
virtual control elements, such as switches, are projected into the driver's field 
of vision by means of the head-up display, so that real switches can be 
eliminated. 

A disadvantage of the aforementioned control device according to DE 196 53 
595 CI is however, furstly the fact that the camera photographs not only the 
driver's hand and the control elements to be operated, but also, for example, 
parts of the centre console, which has a distracting effect when superimposed 
in the driver's field of vision. Secondly, the driver's hand may hide the control 
element to be operated from the perspective of the camera, and thus the 
superimposition of the driver's hand in his field of vision does not enable 
satisfactory operation. 
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SUMMARY OF THE INVENTION 

The object of the invention is thus to provide an improved control device, with 
no distracting parts of the picture are superimposed in the driver's field of 
vision and enabling a good ergonomic operation. 

This object is achieved by a control device, particularly for low-distraction 
operation of switches in a vehicle, which comprises a sensor for detecting the 
position of an input element manually guided by an operator, a visual display 
unit arranged in the field of vision of the operator, for display of at least one 
virtual element, an evaluation unit, the input of which is connected to the 
sensor, for determining the position of the manually-guided input element, and 
an image sensor, the input of which is connected to the evaluation unit and the 
output of which is connected to the display unit, for displaying a virtual image 
indicator in the field of vision of the operator, corresponding to the position of 
the manually-guided control element. 

The invention is based on the general technical rule that the picture of the 
driver's hand taken by the camera is not to be directly superimposed in the 
driver's field of vision, but instead is to be used only to determine the position 
of the driver's hand. Then only a virtual image indicator corresponding to the 
posidon of the driver's hand determined in this way is superimposed in the 
driver's field of vision by means of a visual display unit, with distracting visual 
background signals in the picture taken by the camera, such as parts of the 
centre console, being hidden. 

The sensor detects the position of an input element manually guided by the 
operator, whereby the input element, preferably is the hand or the index finger 
of the operator. It is, however, also possible that the operator holds a separate 
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input element that interacts with the sensor to enable the position to be 
determined. A suitable sensor would advantageously be a camera, particularly 
an infrared camera, an ultrasound sensor, a radar sensor, a capacitive sensor or 
a contact sensor, such as a so-called touch-pad. 

The visual display unit is arranged in the field of vision of the operator to 
display at least one virtual control element and the virtual image indicator, 
whereby the display unit preferably consists of a "head-up display" that can 
project a picture directly into the field of vision of the operator. 

In a preferred embodiment of the invention, the output of the evaluation unit is 
connected to a comparator that compares the position of the manually-guided 
control with the preset position data determined by the evaluation unit and 
<^enerates a control signal relative to the result of the comparison. The preset 
position data in this case preferably corresponds to the positions of die virtual 
switch elements to be individually actuated, «o that to trigger a switch acuiation 
the user needs only to bring the manually-guided element into a required spatial 
area. 

The output of the comparator is preferably connected to a loudspeaker to 
acoustically indicate a possible and/or implemented actuation of a virtual 
control. In this way it is, for example, possible to provide the operator with an 
acoustic feedback in that an acoustic signal is output when the manually-guided 
element is in an acmation position before a switch element. By hearing the 
acoustic feedback the operator then, for example, knows that all that is required 
is to press the manually-guided input element, such as the index finger, forward 
in order to activate the control element. After activating the control element a 
further acoustic signal which may differ from the first acoustic signal is then 
output to indicate the successful actuation of the control element. 
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In another embodiment the output of the evaluation unit is connected to a 
limiter that hmits the position data, determined by the evaluation unit, of the 
manually-guided input element to preset values. In this way, for example, it is 
possible to prevent the virtual image indicator superimposed in the field of 
vision of the operator moving out of the field of vision in the event of a 
sweeping hand movement by the operator. The virtual image indicator then 
instead remains at the boundary of the field of vision to facilitate re-positioning 
by the operator. Preferably, the output of the limiter is also connected to a 
loudspeaker to provide the operator with acoustic feedback so that the 
loudspeaker outputs an acoustic warning signal if the hand of the operator 
moves out of the operating area. 

Further advantageous embodiments of the invention are identified in the 
subclaims and explained in the following together with a description of the 
specific embodiments in connection with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a perspective view of the inside of a vehicle showing the control 
device in accordance with the invention. 

Figure 2 is block diagram of the control device in accordance with the 
invention. 

Figure 3 is another embodiment of the invention using a touch-pad as a 
sensor. 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The perspective view shown in Figure 1 is of the inside of a vehicle with a 
driver 1 and a camera 2, arranged in the headlining above the centre console, 
which is directed downwards to an operating area 3 located in the area of the 
centre console. The the operating area 3 unlike normal control devices contains 
no actual switch elements or control elements, but instead serves merely as a 
contact area for an index finger 4 of the driver 1 acting as an input element. 
The picture of the operating area 3 taken by the camera 2 therefore essentially 
contains the position of the driver's index finger 4 as information. 

Furthermore, the control device has a so-called head-up display 5, shown only 
schematicaUy in Figure 2. Details of a head-up display 5 of this kind are given 
in German unexamined application DE 196 20 199 Al. No detailed description 
of the head-up display 5 is therefore given, but it is referred for these details to 
DE 196 20 199 Al, which is incorporated by reference. The head-up display 5 
superimposes virtual control element 6 in the field of vision of the driver 1, 
1 whereby the virtual control element 6 can, for example, be a virtual switch. The 
head-up display 5 also projects a virtual image indicator 7 into the field of 
vision of the driver 1. 

The output of the camera 2 is connected to an image analyser 8 which acts as 
an evaluation unit, that analyses the picture of the driver's hand 1 taken by the 
camera 2 and determines the spatial position of the end of the index finger 4 
and outputs this in the form of X, Y, Z position data. 

A limiter 9 is fitted in circuit after the image analyser 8 and compares the 
position data X, Y, Z of the end of the index finger 4, calculated by the image 
analyser 8, with the preset position data which forms the boundary of the 
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operating area 3. If the driver 1 moves the index finger 4 out of the operating 
area, the X, Y, Z position data of the end of the index finger 4 is hmited to the 
limit values Xlim^ Yum, Zlim at the boundaries of the operating area 3. 

Furthermore, the output of the limiter 9 is connected to a loudspeaker 10 that 
outputs an acoustic warning signal if the hand of the user leaves the operating 
area 3. 

The output of the limiter 9 is connected to a comparator 1 1 that compares the 
limited position data Xlim> Yum. Zlim with the preset position data 
corresponding to the virtual positions of the control element 6. If the driver 1 
moves his index finger 4 such that the end of the index finger 4 is within the 
area of the virtual positions of the sv^itch elements 6, the comparator 1 1 outputs 
a corresponding control signal CONTROL to trigger the function associated 
with the particular control element 6. In this case, the comparator 11 also 
activates the loudspeaker 10 to generate an acoustic feedback signal that 
indicates a successful actuation of the particular control element 6 to the driver 
1. Furthermore, the output of the comparator 11 is connected to an image 
sensor 12 that controls the head-up display 5 for display of the virtual control 
elements 6. That control element 6 activated by the driver 1 is being displayed 
in a different colour after successful activation, so that the driver 1 has not only 
an acoustic feedback signal from the loudspeaker 10 but also a visual feedback 
signal by the representation of the control element 6 in the head-up display 5 in 
a different colour. 

The control device further has an image sensor 13 that on the input end is 
connected to limiter 9. The image sensor 13 controls the head-up display 5 to 
display the virtual image indicator 7 relative to the Xlim^ Ylim, Zlim position 
data limited by the limiter 9. 
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The embodiment shown in Figure 3 is identical to those embodiments shown in 
Figures 1 and 2, with one exception as descriped below. Therefore same 
reference numerals are used, which have the same meaning in all figures. 

The particular feature of the embodiment shown in Figure 3 is that the sensor 2 
is not a camera but a contact-sensitive touch-pad that outputs the position at 
which the driver 1 touches the touch-pad 2. 

The invention is not limited to embodiments described herein. A number of 
variants and modifications are also conceivable, that make use of the concept in 
accordance with the invention and therefore fall within the scope of protection. 
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CLAIMS 



1 . Control device, particularly for low-distraction operation of switches in a 
vehicle, comprising 

a sensor (2) for detecting the position of an input element (4) manually guided 
by an operator and 

a visual display unit (5) arranged in the field of vision of the operator, for 
display of at least one virtual element (6), 

an evaluation unit (8), the input of which is connected to the sensor (2), for 
determining the position of the manually-guided input element (4), and 

an image sensor (13), the input of which is connected to the evaluation unit (8) 
and the output of which is connected to the display unit (5), for displaying a 
virtual image indicator (7) in the field of vision of the operator, corresponding 
to the position of the manually-guided control element (4). 

2. Control device according to Claim 1, wherein the output of the 
evaluation unit (8) is connected to a comparator (9) that compares the position 
determined by the evaluation unit (8) with preset position data and generates a 
control signal (CONTROL) relative to the result of the comparison. 

3. Control device according to Claim 2, wherein for acoustic confirmation 
of a possible and/or implemented actuation of a virtual control element (6), a 
loudspeaker (10) is provided, whereby the loudspeaker (10) is connected to the 
comparator (9) for activation. 



4. Control device according to anyone of the preceding claims, wherein the 
output of the evaluation unit (8) is connected to a limiter (9) that limits the 
position data determined by the evaluation unit (8) to preset values. 

5. Control device according to Claim 4, wherein the output of the limiter 
(9) is connected to a loudspeaker (10) and that an acoustic waming signal is 
output if the preset position data is reached or overshot. 

6. Control device according to anyone of the preceding claims, whereby 
sensor (2) is an optical sensor, an ultrasound sensor, a radar sensor, a capacitive 
sensor or a contact sensor. 

7. Control device according to anyone of the preceding claims, whereby 
sensor (2) is a camera, an infrared camera or a touch-pad. 

8. Control device according to anyone of the preceding claims, wherein the 
visual display unit (5) is a head-up display. 
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